Generating Descriptions of Spatial Relations between Objects in Images
نویسندگان
چکیده
We investigate the task of predicting prepositions that can be used to describe the spatial relationships between pairs of objects depicted in images. We explore the extent to which such spatial prepositions can be predicted from (a) language information, (b) visual information, and (c) combinations of the two. In this paper we describe the dataset of object pairs and prepositions we have created, and report first results for predicting prepositions for object pairs, using a Naive Bayes framework. The features we use include object class labels and geometrical features computed from object bounding boxes. We evaluate the results in terms of accuracy against human-selected prepositions.
منابع مشابه
مدلسازی روابط توپولوژیک سه بعدی فازی در محیط GIS
Nowadays, geospatial information systems (GIS) are widely used to solve different spatial problems based on various types of fundamental data: spatial, temporal, attribute and topological relations. Topological relations are the most important part of GIS which distinguish it from the other kinds of information technologies. One of the important mechanisms for representing topological relations...
متن کاملColor Object Recognition based on Spatial Relations between Image Layers
The recognition of complex objects from color images is a challenging task, which is considered as a keystep in image analysis. Classical methods usually rely on structural or statistical descriptions of the object content, summarizing different image features such as outer contour, inner structure, or texture and color effects. Recently, a descriptor relying on the spatial relations between re...
متن کاملA Corpus of Natural Multimodal Spatial Scene Descriptions
We present a corpus of multimodal spatial descriptions, as commonly occurring in route giving tasks. Participants provided natural spatial scene descriptions with speech and abstract deictic/iconic hand gestures. The scenes were composed of simple geometric objects. While the language denotes object shape and visual properties (e.g., colour), the abstract deictic gestures “placed” objects in ge...
متن کاملObject-Oriented Method for Automatic Extraction of Road from High Resolution Satellite Images
As the information carried in a high spatial resolution image is not represented by single pixels but by meaningful image objects, which include the association of multiple pixels and their mutual relations, the object based method has become one of the most commonly used strategies for the processing of high resolution imagery. This processing comprises two fundamental and critical steps towar...
متن کاملA Study on how Humans Describe Relative Positions of Image Objects
Information describing the layout of objects in space is commonly conveyed through the use of linguistic terms denoting spatial relations that hold between the objects. Though progress has been made in the understanding and modelling of many individual relations, a better understanding of how human subjects use spatial relations together in natural language to is required. This paper outlines t...
متن کامل